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About This Brief 


The Arizona State Board of Education (Board), the 
Arizona Department of Education (Department), The 
Center for Assessment, Helios Education Foundation, 
Abt Associates, and the ASU Helios Decision Center for 
Educational Excellence partnered to study the impacts 
that the pandemic has had—and continues to have— 
on the educational experiences and achievements of 
students in the state. 


To date, the Board and Department have published 
extensive data and findings on the how the pandemic 
has impacted student growth, proficiency, and 
enrollment and mobility. These findings have been 
reviewed, discussed, and presented at the Board Fall/ 
Winter 2021 meetings. The Board has also published 
preliminary findings on how the pandemic has impacted 
students who are English Learners. 


This brief provides a condensed analytical look at key 
data points and indicators derived from the reports 
and data presentations publicly available on the 
Department’s website (see COVID-19 Impact Data and 
Reporting). The insights it contains can inform policy 
and decision-making as the state works to facilitate 

a multi-year recovery from the large and widespread 
impacts of the pandemic. 
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Introduction 


As the pandemic surges into its third academic year, 
students, families, and educators in Arizona continue to 
weather the disruption and uncertainty that characterize 
the educational experience in the COVID-19 era. 


This effect of the pandemic on learning can be likened to 
a sustained academic headwind that is not only slowing 
the rate at which students progress, but also, because 

of that slower rate, is limiting the overall educational 
progress students can make. This results in lower rates 
of student growth and attainment throughout the state, 
with some demographic subgroups experiencing notably 
worse declines than others. 


Educators, school staff, and school and district leaders 
have all endeavored to meet the educational and basic 
needs of their students and school communities. 
However, the illness, loss, and economic hardship that 
the pandemic is inflicting—in addition to the disruptions 
of school settings—continue to negatively affect 
students’ educational opportunities and achievement, 
despite the return to in-person learning. 


This brief highlights select state-level findings from 
in-depth and collaborative research undertaken to 
analyze the impacts of the pandemic on student growth, 
proficiency, and enrollment and mobility. 


Summary of Main Research Findings 


Students in the early grades experienced larger academic 
impacts than those in older grades. Compared to the 
older grades, proficiency and growth data for the K-5 
grades show a greater impact. This is particularly true in 
English Language Arts (ELA), a finding that is especially 
significant given the importance of early literacy fora 
student’s later success. 


Although student achievement in both Mathematics and 
ELA has been impacted to a large degree, the impact 

on Mathematics has been greater. While both subject 
areas see large and broad impact across all demographic 
subgroups and achievement levels, overall impacts 

on students’ Mathematics achievement were larger. 


During the 2020-2021 school year, more students than 
ever before moved among Arizona schools. At the same 
time, fewer students enrolled. Student mobility during 
the school year represents a disruption in learning. As 
such, it contributes to a better understanding of student 
achievement gaps. 


English Learners are struggling more than their English- 
language proficient peers, continuing a trend that 
predates the pandemic. The percentage of English 
Learners (EL) who achieved proficiency fell six points 
in ELA and seven points in Mathematics. The state saw 
declines in proficiency percentages across the board— 
38 percent in ELA and 31 percent in Mathematics in 
2020-2021, compared to 42 percent in ELA and 42 
percent in Mathematics in 2018-2019, when looking 
at the Longitudinal COVID Impact data. 


About the Data 


The COVID-19 impact study included proficiency and 
growth data from the 2020-2021 statewide assessment 
(AzM2), as well as statewide student enrollment, student 
mobility, and AZELLA (Arizona English Language Learner 
Assessment) assessment data. 


Note that comparing student data gathered during 
the 2020-21 school year to data from pre-pandemic 
years is essential to any effort to analyze the impact 
of the pandemic on measures of student proficiency 
and growth. However, because of pandemic-related 
disruptions, student testing participation rates in 
Spring 2021 were lower than they are traditionally. 
This variation impacts comparative analyses, and any 
inferences drawn from such analyses must be made 
with caution. 


To learn more about the data and methodologies used, 
please see the COVID-19 Impact Reporting resources 
available at: https://www.azed.gov/accountability- 


research/data 
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Pandemic Impacts on Student Learning 
Across Grade Levels 


The overall impact of the pandemic on education in 
Arizona is significant. Compared to pre-pandemic years, 
analyses of student growth and proficiency from the 
2020-2021 school year reveal widespread decreases. 
Similarly, overall enrollment in Arizona schools has 
decreased, while student mobility during the school 
year—which can be understood as an academic 
disruption—increased meaningfully as more students 
withdrew from school or transferred. 


Notably, statewide assessment participation rates in 
2020-2021 were lower than historical averages. For ELA, 
84 percent of Arizona students in Grades 5 through 10 
were tested; for Mathematics, 86 percent of students 
were tested. A typical year would expect 95 percent of 
students tested on ELA and Mathematics. 
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Overview: Measures of Impact 


Growth: Arizona uses the Student Growth 
Percentile (SGP) model, in collaboration 

with Dr. Damian Betebenner (Center for 
Assessment), which assesses academic growth 
over one school year regression that links 
current year scores with the scores from the 
immediate prior year(s). Skip year regression 
was used in the 2020-2021 school year due to 
missing assessments in 2019-2020. Student 
growth was also compared to the 2018-2019 
peer group who had not faced the pandemic 
to help measure the impact of COVID-19. 
More information on the methodology can 

be found in the growth resources on the 
Department’s website. 








Proficiency: Compared to historical testing 
rates, there are substantial differences 

in the student population tested in 2020- 
2021. These differences, in addition to 
student mobility and enrollment patterns, 
must be accounted for and acknowledged 

in any interpretation the proficiency data. 
Accordingly, the Board’s analyses used match_ 
rates to measure the student populations’ 
effect on assessment outcomes. 


Enrollment and Mobility: Enrollment is 
examined using a unique count of students— 
students are counted only once—and 
compared to historical trends. Mobility is 
analyzed using school withdrawal data. 





Statewide Decreases in Student Growth During the Pandemic Are Unprecedented 


Student growth has been dramatically impacted by the pandemic. Across grade levels, significant decreases were 
observed in both ELA and Math growth, with the overall decreases in Math exceeding those in ELA. 








ELA Mathematics 
Grade 

2019 2021 Loss Impact 2019 2021 Loss Impact 
5 50.0 (74,441) 32.0 (58,823) -18.0 Large 50.0 (74,508) 26.0 (59,817) -24.0 Large 
6 50.0 (74,522) 36.0 (60,667) -14.0 Moderate 50.0 (74,700) 24.0 (61,592) -26.0 Severe 
7 50.0 (72,608) 33.0 (60,989) -17.0 Large 50.0 (72,716) 30.0 (62,032) -20.0 Large 
8 50.0 (72,017) 37.0 (61,897) -13.0 Moderate 50.0 (64,000) 31.0 (62,906) -19.0 Large 
10 50.0 (52,181) 36.0 (50,370) -14.0 Moderate NA NA NA NA 


Baseline growth in each grade and content area has a mean/median of 50 (number of students shown in parenthesis). The extent to which growth from 2019 to 2021 is below 50 indicates decreased student 
learning due to the pandemic. Impact strength is defined as: Improvement (>5), Moderate to None (-5 to 5), Moderate (-15 to -5), Large (-25 to -15), Severe (<25). 


Student Proficiency Has Decreased Statewide Across Grade Levels 


For all grade levels, ELA and Mathematics proficiency are down compared to pre-pandemic school years. Math proficiency 
is down 11 percent for all tested students, ELA is down four percent for all tested students. The largest, and perhaps most 
troubling, proficiency decrease was observed in ELA for grades 3 to 5. Given the importance of early literacy for student 
success, the impact on future outcomes for these students may be consequential. 
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Student Enrollment Has Decreased Statewide While Student Mobility Has Increased 


Overall, student enrollment declined across Arizona during the 2020-2021 school year. In particular, the data show 
greater pandemic-related decreases in enrollments in the elementary and middle school grade levels and especially 
in Kindergarten, Grade 5, and Grade 6. 
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Similarly, students in the younger grades were more likely to withdraw or transfer to another school than students 
in the older grades, in keeping with an overall increase in student mobility during the pandemic. 
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How the Pandemic Impact Varies for Different Student Groups 


Before the pandemic disrupted learning for students in Arizona, there were prominent disparities and achievement 

gaps among students depending on their ethnicity, family income, disability status, and English Learner status. As might 
be expected, the severity of the pandemic impact on student growth varies; Hispanic and African American students, 
for example, are more impacted than their peers when it comes to growth in Mathematics. It is notable that, despite this 
variability, all groups were at least moderately impacted in ELA and all experienced large or severe impacts in Math. 


Student Population Group ELA Impact Math Impact 





All Students Moderate” Large 


English Learner Status 


English Learners Large Large 
Non-English Learners Large’ Large 
American Indian or Alaska Native Large Severe 
Asian Moderate Large 

African American Moderate Severe 
Hispanic or Latino Large Severe 
Multiple Races Moderate Large 

Native Hawaiian or Pacific Islander Large Severe 
White Moderate Large 


Family Income 


Income Eligibility 1 and 2 Large Severe 
Not Income Eligible Moderate Large 


Disability Status 


Students without Disabilities Large Large 








Students with Disabilities Moderate Large 


Baseline growth in each grade and content area has a mean/median of 50. The extent to which growth from 2019 to 2021 is below 50 indicates decreased student learning due to the pandemic. Impact strength 
is defined as: Improvement (>5), Moderate to None (-5 to 5), Moderate (-15 to -5), Large (-25 to -15), Severe (<25). 


*The unadjusted difference for All Students is -15; it is -15.4 for English Learners; and it is -15.1 for Non-English Learners. The cut score of -15 is between Moderate and Large. Any difference close to or around 
-15, could be considered either as “Moderate” or “Large.” 


“COVID-19 continues to expose glaring inequities, and in turn, we need to elevate our response. 
The first course of action in shaping that response is to fully understand the impact of the pandemic 
on students, teachers, families, schools, and communities. The partnership represented in this 
research is an important one, as it brings together public and private sectors in ways that are 
important to meet the immediate needs of students and educators now, and to create sustained 
and systematic approaches to improving the conditions that improve student learning over time.” 





Paul J. Luna, President and CEO, Helios Education Foundation 
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Assessment data also indicate that all student population groups, without exception, have seen decreases in both ELA 
and Math proficiency. Some population groups, though, have seen greater decreases; Native American, African American, 
and Hispanic students registered notable proficiency decreases (from 2019 to 2021), the lowest proficiency rates among 
all ethnic groups, and proficiency rates well below the average for all students. 


Students without Disabilities M@FY21 Wm FY19 
Students with Disabilities 
Non Income Eligibility 1 & 2 
Income Eligibility 1 & 2 
White 

Native American 

Multi Race 

Pacific Islander 

Hispanic 


Asian 65% 


70% 
African American 
Non English Learner 
English Learner 


All Students 
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Despite the overall one percent decrease in enrollment in 2020-2021, most student population groups did not see 
substantial declines. The exception to this trend is students experiencing homelessness, who, as a group, saw a 20 
percent decrease in enrollment. This decrease is likely attributable to their status. By contrast, enrollment for certain 
student populations—African American, American Indian or Alaska Native, Asian, and Native Hawaiian or Pacific Islander 
subgroups increased in 2020-2021. 








Subgroup FY19 FY20 FY21 Percent Change | Percent Change 
Enrollment Enrollment Enrollment v0) ho e040) 2020 - 2021 
All Students 1,238,727 1,239,759 1,225,694 0 -1 
English Learners 90,254 80,081 80,980 -11 1 
Ethnicity 
African American 64,098 65,166 72,107 2 11 
American Indian or Alaska Native DO A0T 51,700 55,394 -1 a 
Asian 32,823 33,247 36,523 1 10 
Hispanic or Latino 568,362 562,951 565,463 -1 0 
Multiple Races 65,572 67,728 47,485 3 -30 
Native Hawaiian or Pacific Islander 3,361 3,413 4,870 2 43 
White 452,226 455,429 447,301 1 -2 
Status 
Students with Disabilities 172,334 172,274 170,798 0 -1 
Foster Care 6,464 7,480 7,830 16 5) 
Homeless 19,794 17,458 13,910 -12 -20 
Military 9,348 10,852 11,323 16 4 
Income Eligibility 582,544 600,265 551,907 3 -8 














“This research is a critical tool in assessing the pandemic’s impact on students and will help education 
professionals shape needed efforts to help kids catch up academically, socially, and emotionally. The 
data also highlights the critical need for ongoing and sustainable support for our public schools and 
the students they serve. Arizona students are ready to learn, and our teachers are ready to provide 
every student with the academic and mental health support they need. After two tumultuous school 
years, our school communities are working overtime to accelerate learning and help students succeed. 
It is the job of the state to support them in their work.” 





Kathy Hoffman, Superintendent, Arizona Department of Education 


Helios Education Foundation | Arizona State Board of Education | Arizona Department of Education 7 





8 Increased Disruption, Decreased Progress 





A Call to Action for Arizona 


The pandemic remains a prominent and influential factor in the educational experience for students in Arizona. As the 
recent resurgence of COVID-19 cases in the state and across the country has signaled—and as the disruptions to school, 
professional, social, and family life continue—the ability for schools to resume pre-pandemic “normal” operations 
remains elusive. 


The return to in-person learning has been, for many, a positive step toward normality and a welcome opportunity for 
students to reengage with their schools, teachers, and peers. But the resumption of in-person learning in Arizona, while 
beneficial, is not sufficient to counteract the adverse effects of the pandemic on student achievement and growth, much 
less the social and emotional difficulties many students are experiencing. 


Given the extent of the disruption and loss caused by the pandemic, Arizona will need to sustain a multi-year recovery 
effort structured around intentional strategies to address incomplete learning, accelerate student progress, and provide 
targeted supports to students and educators. Policies and programs that support the implementation and sustainability 
of such strategies are also vitally important. 


To inform these efforts, it is essential that entities throughout the state continue to prioritize sustained, collaborative 
studies of the pandemic impact on student learning and outcomes. A deep and nuanced understand of the ways in which 
students have struggled against the academic headwind and lost ground in their learning will be necessary to inform the 
development of effective long-term strategies and interventions. 
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